# Efficient attention mechanism

Tweety-7b-dutch is a foundational large language model specialized in Dutch, based on the Mistral architecture, optimized for Dutch text processing with a Dutch tokenizer.

Mistral-7B-Instruct-v0.2 is an instruction-finetuned large language model improved from Mistral-7B-Instruct-v0.1, compressed to 2% sparsity using Wanda pruning method while maintaining competitive performance without retraining.

MPT-7B-Instruct-8k is a model for long-format instruction following, especially good at answering questions and summarizing long documents.

Chinese pre-trained model based on BigBird architecture, supporting 4096-length context processing